One of the most challenging problems in color image processing is adjusting the color gains of a system to compensate for variations in illumination spectra incident on an imaging sensor, also known as “white balance”. The human eye and brain are capable of “white balancing.” If a human observer takes a white card and exposes it under different kinds of illumination, it will look white even though the white card is reflecting different colors of the spectrum. If a person takes a white card outside, it looks white. If a person takes a white card inside and views it under fluorescent lights, it looks white. When viewed under an incandescent light bulb, the card still looks white. Even when placed under a yellow light bulb, within a few minutes, the card will look white. With each of these light sources, the white card is reflecting a different color spectrum, but the brain is smart enough to make it look white.
Obtaining the same result with a camera or other imaging device is harder. When the white card moves from light source to light source, an image sensor “sees” different colors under the different lights. Consequently, when a digital camera is moved from outdoors (sunlight) to indoor fluorescent or incandescent light conditions, the color in the image shifts. If the white card looks white when indoors, for example, it might look bluish outside. Alternatively, if it looks white under fluorescent light, it might look yellowish under an incandescent lamp.
The white balance problem stems from the fact that spectral emission curves of common sources of illumination are significantly different from each other. For example, in accordance with Plank's law, the spectral energy curve of the sun is shifted towards the shorter wavelengths relative to the spectral energy curve of an incandescent light source. Therefore, the sun can be considered to be a “blue-rich” illuminator while an incandescent bulb can be considered to be a “red-rich” illuminator. As a result, if the color processing settings are not adjusted, scenes illuminated by sunlight produce “bluish” imagery, while scenes illuminated by an incandescent source appear “reddish”.
In order to compensate for changes in illumination spectra, the gains of color processing systems and/or imagers should be adjusted. This adjustment is usually performed to preserve the overall luminance (brightness) of the image. As a result of proper adjustment, gray/white areas of the image appear gray/white on the image-rendering device (hence the term “white balance”). In the absence of specific knowledge of the spectra of the illumination source, this adjustment can be performed based on an analysis of the image itself to obtain color balance information, i.e., information about the luminance of colors in the image.
One conventional approach to computing the proper adjustment to the color gains is based on the premise that all colors are represented equally in complex images. Based on this assumption, the sums of all red, green and blue components in the image should be equal (in other words, the image should average to gray). Following this approach, the overall (average over the entire image) luminance Y, and red (R_avg), green (G_avg) and blue (B_avg) components are evaluated. The color gains (G_red, G_Green, G_blue) are then selected so that Y=G_red*R_avg=G_green*G_avg=G_blue*B_avg.
This conventional approach produces reasonable color rendition for images containing a large number of objects of different colors or large gray areas. However, if the image contains any large monochrome regions, the conventional approach fails. This is the case in many practical situations. Typical examples of such images with a large area having only one color include landscapes in which a significant portion of the image is occupied by either blue sky or green vegetation. Other examples include close-up images of people, wherein flesh tones occupy a significant portion of the image. Yet another example is a non-gray wall serving as a background of the image.
In all of the above examples with large monochrome areas, the averages of the color components of the image would not be equal. An adjustment of the gains based on such proportions would not produce a properly white-balanced image. In other words, the conventional approach to white balancing an image does not correctly compensate if an image includes large monochrome regions.
As depicted in FIG. 1, other procedures for white balancing subdivide an image frame into a plurality of subframes, and each subframe is analyzed to determine if that subframe is predominantly monochromatic other than gray or white. If so, that subframe is excluded from the computation of the gain adjustments. As a result, the white balance process is performed using only multicolored and/or gray subframes. As shown in FIG. 1, each subframe marked with an “X” is determined to be monochromatic, and is excluded from the white balancing operation.
However, the use of such methods in a system often requires large computing and memory resources. Implementation in a system which supports different frame sizes also presents difficulties. It would be advantageous to have improved white balancing techniques.